SILT: Efficient transformer training for inter-lingual inference

The ability of transformers to perform precision tasks such as question answering, Natural Language Inference (NLI) or summarizing, has enabled them be ranked one the best paradigms address Processing (NLP) tasks. NLI is scenarios test these architectures, due knowledge required understand complex sentences and established relationships between a hypothesis premise. Nevertheless, models suffer from incapacity generalize other domains difficulties face multilingual interlingual scenarios. leading pathway in literature issues involve designing training extremely large but this causes unpredictable behaviors establishes barriers which impede broad access fine tuning. In paper, we propose new architecture called Siamese Inter-Lingual Transformer (SILT). This able efficiently align embeddings for Inference, allowing unmatched language pairs processed. SILT leverages siamese pre-trained multi-lingual with frozen weights where two input attend each later combined through matrix alignment method. experimental results carried out paper evidence that allows reduce drastically number trainable parameters while inter-lingual achieving state-of-the-art performance on common benchmarks.